Method for recognizing objects in images

ABSTRACT

A method for recognizing objects in images is disclosed. The method of the present invention comprises the following steps. First, acquire a digital image. Then, select one or more objects from the image according to a certain characteristic. Next, generate an x-axis histogram and/or a y-axis histogram from the segmented image. Then, find the zeroes and maxima for the x-axis histogram and/or the y-axis histogram and use the polynomial regression analysis to determine the shape, shape and location of each of the objects in the segmented image according to the zeroes and maxima. If the two curves linking two zeroes and one maximum in the x-axis histogram and the y-axis histogram are two sloped line, the corresponding object may be determined to be a triangle. If each of the four curves linking two zeroes and two maxima is a line, the corresponding object may be determined to be a rectangle.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention generally relates to a method for recognizing objects inimages. More particularly, the invention relates to a method by whichthe shape, size and position of an object may be accurately and swiftlydetermined.

2. Description of the Prior Art

The object recognition system in the prior art includes the followingfive steps: (1) Acquire an image by a still camera or a similar device(2) Segment the image (3) Extract objects from the segmented image (4)Abstract the extracted objects (5) Classify the abstract objects.

Captured input images are typically composed of color or gray-scalepixel data in digital format, arranged as two-dimensional matrices inthe x (horizontal) and y (vertical) directions. Images may containthousands or millions of individual pixels.

At the lowest level of abstraction (no abstraction at all), objects canbe modeled as whole images and compared, pixel by pixel, against a rawinput image stored in a pattern matching database. However, most objectrecognition systems use various methods for segmenting images andextracting, abstracting, and classifying image objects to improverecognition speed and/or accuracy (Krumm, U.S. Pat. No. 7,092,566).

In particular, color histograms have been used in various steps of theprocess, primarily to improve object recognition speed in color images.

In U.S. Pat. No. 7,020,329, Prempraneerach, et al., described of usingcolor histograms to segment a color image into a plurality of regions byconverting the image into a three-dimensional color space and thengenerating a color histogram for each dimension in the color space. Usethe histograms to generate a plurality of connecting boxes in thethree-dimensional color space and computing a normalized variance valuefor each connecting box to form clusters of connecting boxes. Map thesegmented and clustered pixels back to the image domain to extractsegmented regions from the image, and classifying clustered pixels inthe image domain to recognize objects in the image, which correspond toareas of the image with consistent color characteristics. Recommendedclassifying methods include neural networks (with adaptive templatematching), frequency-sensitive competitive learning, rival-penalizedcompetitive learning and statistical classifiers.

Prempraneerach, et al., described the proposed clustering technique asmore efficient than prior iterative clustering techniques. In addition,they report that the proposed segmentation method further reducescomputation time by processing each pixel only once, to create theintensity, hue, and saturation histograms which are then used forclustering.

However, the segmentation method does not make use of prior informationconcerning known image color content and the color histograms which arecreated do not maintain geometry information. As a result, the methodstill requires a series of complex computations to complete the objectrecognition process. First, the input image is filtered with an edgepreserving filter to smooth the color image and reduce discontinuitiesin the subsequently derived color histograms, then all of the pixels inthe image are converted from RGB to LUV and LUV to IHS color spacevalues using a set of eleven equations, IHS histograms are computed forall of the pixel data, the histograms are filtered to remove highfrequency noise, and each histogram is searched for valleys to createconnecting boxes (by convolving each histogram with a Gaussian kernel orGaussian filter).

Clusters of connecting boxes are then formed by computing the normalizedvariance of pixel values within each connecting box and connecting boxesare linked into tree-like structures based upon normalized variancevalues. Connecting boxes which have local minimum normalized variancevalues serve as root nodes for each tree-like structure. The remainingconnecting boxes are linked as branch nodes to root nodes using asteepest gradient descent algorithm.

For the method to work properly, homogeneous regions in the image mustcluster into well defined intervals in the three histograms. However, inpractice, actual image color variations may be difficult to distinguishfrom color variations that result from either noise or the nonlineartransformation from RGB to IHS color space.

In U.S. Pat. Nos. 7,092,566 and 6,952,496 and 6,611,622, Krumm describesusing color histograms for both representing and classifying segmentedregions in an input image. The described object recognition method firstcreates model histograms of objects to be recognized and then segmentsan input image to extract regions which likely correspond to the objectsto be recognized, derives histograms from the segmented regions, andthen compares the derived histograms with the stored model histograms.Similarity measures between input and model histograms that exceed aprescribed threshold indicate that an input object matches a modelobject. Matching input histograms may also be added to a database ofmodel histograms for the given object.

The described method creates model and input image histograms bydetermining the actual RGB colors exhibited by the pixels in a model orinput image region, dividing the overall range of actual pixel colorsinto a series of discrete color ranges or quantized color categories,assigning each pixel of the extracted model or input image region to thequantized color category into which the actual color of the pixel falls,and establishing a count of the number of pixels assigned to eachquantized color category. In a preferred embodiment, RGB pixel valuesare quantized into 27 color categories. Input image and model histogramsare compared by comparing the pixel counts from each quantized colorcategory of the input image and model histograms. Model histograms mustbe derived from a prefatory image which is similar to input images fromwhich objects are to be recognized. Regions of the image from whichmodel histograms are derived are also used in subsequent input imagesfor extracting objects to be recognized.

In a preferred embodiment, model images are segmented by analyzing atime sequence of images from the same imaging device, determining astatic background image by identifying pixel values that do not changeappreciably in the time sequence of images, producing a foreground imageby subtracting the background image from a subsequent image, andsegmenting the foreground image into object regions by identifyinggroups of smoothly varying pixel values.

However, the method is primarily useful for tracking known objects in atime sequence of images. In addition, the method still requires asignificant amount of processing time for creating model histograms andcomparing input image histograms to model histograms. The colorhistogram generation technique used does not preserve geometry, just anaccounting of the number of image pixels of given colors. As a result,actual object shape, size, and location cannot be determined. Finally,similar histograms for different objects may lead to inaccurate objectrecognition results.

In U.S. Pat. Nos. 6,532,301 and 6,477,272, Krum, et al., describe usingco-occurrence histograms to represent and identify the location of amodeled object in a search image.

The process starts by creating model images of the object and thencomputing a co-occurrence histogram (CH) for each of the model images.Model images are created by capturing sets of images of the object to beidentified from viewpoints spaced at equal angles from each other aroundthe object and at various distances. Co-occurrence histograms arecomputed by identifying every possible unique, non-ordered pair ofpixels in the model image and generating counts of pairs of pixels whichhave colors that fall within the same color range and which areseparated in distance by the same distance range.

Next, search windows, of prescribed size, are generated from overlappingportions of the search image, and a CH is computed for each of thesearch windows, using the technique and pixel color and distance rangesestablished for the model image co-occurrence histograms.

Finally, each model image CH is compared to each search window CH toassess their similarity. Every search window CH that matches a modelimage CH, as indicated by a similarity value which exceeds a thresholdvalue, is designated as potentially containing the object to berecognized. The location of the recognized object is then determined tobe within the single search area with the largest similarity measure,among all search areas designated as potentially containing the objectto be recognized. The location of the recognized object can be furtherrefined by iteratively moving the identified search area up, down, left,or right by one pixel location and then re-computing the search windowCH and re-comparing the search window CH with each model CH, to findpotentially higher similarity measures. The system and process requiresthat search window size, color ranges, and distance ranges be chosenbefore image searching begins.

Krum describes several advantages of the proposed method. In particular,Krum states that co-occurrence histograms are an effective way torepresent objects for recognition in images. Keeping track of pairs ofpixels, which have matching colors and a given distance between them,allows a variable amount of geometry information to be added to aregular color-only histogram. In turn, considering both color andgeometry, allows the object recognition process to work in spite ofconfusing background clutter and moderate amounts of occlusion andobject flexing.

However, creating a model image database for histogram matching takes asignificant amount of time. Computing co-occurrence histograms bycomputing distance measures for every possible unique, non-ordered pairof pixels in both the model images and the search image is alsocomputationally expensive.

In addition, the given abstract object representation contains nospecific geometric information other than the distance betweenlike-colored pixels in the model and search images. Informationconcerning actual object shape, size, and location is lost. Theresulting object location determination is not precise, and subsequentiterative refinement of the determined location is computationallyintensive.

Further, method parameters, such as search window size, affect objectrecognition accuracy, and images must be scaled to handle overall sizedifferences between search and model images.

A useful and relatively well-defined application for object recognitionsystems is real-time traffic sign recognition from moving vehicles. Ingeneral, traffic sign systems must be capable of fast object extractionand accurate object classification.

In U.S. Pat. No. 6,801,638, Janssen, et al. describe a process anddevice for recognizing traffic signs and then displaying them as memoryaids for an observer. Images are captured by an image sensor andanalyzed and classified by classifiers implemented in an informationprocessing unit. A synthetic image of a traffic sign is then generated,stored in a memory unit, and displayed by means of a display unit.

Input images are first searched, by color and/or spatial position, todetermine areas which, with above average probability, could containobjects which are traffic signs. Objects are recognized within thedetermined areas by hierarchically and sequentially classifying theimage areas by separate known characteristics of traffic signs, forexample a correlation process to identify outer shape (circle or square)and inner symbols, with respect to stored characteristic data. Theclassifiers compare logical distance between input object characterizingdata and typical characterizing data sets stored in a memory unit.Objects are recognized when comparison distances fall below setthresholds.

However, the classifiers, so designed, must be trained with severalpasses, to handle variations in image quality due to varying weather andlight conditions. The described classifiers also depend uponcorrelations between input object shape data with shape data stored in amemory unit. In general, correlation-based classifiers can be impreciseand/or slow. Correlation depends upon training, consistency of theviewed environment, and/or quality of the stored shape data.

Improving stored shape data requires extensive training or a largestored database. In turn, a large stored database requires moreprocessing time to complete correlations. As an example, circular andsquare objects appear as varying oval and rectangular shapes fromdifferent viewing angles, which could reduce object recognitionaccuracy.

The method described also depends upon searching the input image forareas which could contain traffic signs based upon color values and/orspatial position.

In U.S. Pat. No. 6,813,545, Stromme describes a system for reminding adriver of the presence of at least one particular traffic sign. Thesystem consists of an imaging unit attached to the vehicle and directedtoward the road ahead of the vehicle, a database which contains at leastone pre-registered traffic sign shape, and an automatic recognition unitfor detecting and identifying traffic signs, in successive images, bysearching images areas which have a shape contained in the database, aselection process between two signs contained in the same image fordetermining the distance between the vehicle and the signs, and a soundand/or visual indicator which signals that an identified traffic sign ispresent on the road ahead of the vehicle.

Input images are captured periodically, based upon the speed of thevehicle. Each input image is analyzed in a shape recognition processor,to detect the presence of traffic sign shapes and traffic sign symbolshapes contained in the shape information database.

In a preferred embodiment, the shape search and recognition unit withinthe system uses conventional image processing methods: simple edgedetection, such as Canny edge detection, followed by simplepixel-by-pixel matching across the processed image. Several views of thesign shapes are stored in the shape matching database. For triangles,circles, or rectangles, symbols contained within the sign are alsoidentified, using pattern or recognition algorithms applied to thedetected shape. Color detection is also carried out to check that adetected shape is effectively a traffic sign.

However, direct image searching methods and, in particular, edgedetection and pixel-by-pixel pattern matching methods are generallyimprecise and relatively slow. In addition, both sign and sign symbolshape and size vary significantly, based upon vehicle position withrespect to a given traffic sign, which further hinders accurate shapematching and object recognition using edge detection and pixel-by-pixelmatching.

In U.S. Pat. No. 5,926,564, Kimura uses histograms for objectrecognition. In his method, scanning is conducted along the x-axis andy-axis and “0-1 pattern” is used for comparison of images. Thedisadvantage of the method is the fact that the shape and size of anobject can not be correctly calculated if there is a misalignmentbetween an image capture device and the object.

From the above, we can see that the methods of the prior art have manydisadvantages and need to be improved.

To eliminate the disadvantages of the methods of the prior art, theinventor has put in a lot of effort in the subject and has successfullycome up with the method of the present invention.

SUMMARY OF THE INVENTION

A first object of the present invention is to provide a method in whichzeroes and maxima are located in an x-axis (horizontal axis) histogramand/or a y-axis (vertical axis) histogram and the polynomial regressionanalysis is employed to swiftly determine the shape, size and locationof each of the objects in the segmented image.

Another object of the present invention is to provide a method in whichone or more objects are selected from the image according to a certaincharacteristic, which may be an RGB characteristic or a characteristicof grayscale or visible frequency spectrum and then the shape, size andlocation of each of the objects in the segmented image are swiftlydetermined so that the method may be used in the determination orrecognition of traffic signs and in other fields.

To reach these objects, the method of the present invention isdisclosed. The method of the present invention comprises the followingeight steps:

Step 1: Acquire a digital image

Step 2: Select one or more objects from the image according to a certaincharacteristic, such as a color-specific characteristic (RGB or IHScolor), a characteristic of grayscale and a spectrum-specificcharacteristic.

Step 3: Generate an x-axis (horizontal axis) histogram and/or a y-axis(vertical axis) histogram from the image of Step 2

Step 4: Locate the zeroes and maxima for the x-axis histogram and/or they-axis histogram

Step 5: Use the polynomial regression analysis to determine the shape ofeach of the objects according to the zeroes and maxima. If the twocurves linking two zeroes and one maximum in the x-axis histogram or they-axis histogram are two sloped line, the corresponding object may bedetermined to be a triangle. If each of the four curves linking twozeroes and two maxima in the x-axis histogram or the y-axis histogram isa line, the corresponding object may be determined to be a rectangle. Ifthe relation between two zeroes and two maxima in the x-axis histogramor the y-axis histogram is in the form of a quadratic function, thecorresponding object may be determined to be a circle or an ellipse.

Step 6: Use the distance between two zeroes in the x-axis histogram orthe y-axis histogram to determine the size of the corresponding object

Step 7: Use the coordinates (locations) of the zeroes in the x-axishistogram or the y-axis histogram to determine the location of thecorresponding object.

Step 8: Select objects from the original image of Step 1 according toanother characteristic to generate another image and then repeat Step 2to 7 to process the new image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Please see FIG. 1, which illustrates the flowchart for the method of thepresent invention. The flowchart includes the following 8 steps.

Step 1: Acquire an image of digital pixel data by a digital imagecapture device. Alternatively, an analog image may first be acquired byan analog image capture device and such analog image may then beconverted to a digital image, which is to be analyzed. Also, suchdigital image capture device or analog image capture device may be adigital/still camera, a video camera or other types of image capturedevices 801.

Step 2: Select one or more objects from the image (segment the image)according to a certain color-specific characteristic of the image 802.In one embodiment, the image is a color image, and the image issegmented by a single color. In another embodiment, the image may be agrayscale image, and the image may be segmented by a known grayscaleintensity value of an object to be recognized in the image. In anotherembodiment, the image may be of a visible frequency spectrum, and theimage may be segmented by such visible frequency spectrum.

Step 3: Then, an X-axis (horizontal axis) histogram and/or a Y-axis(vertical axis) histogram 803 are calculated by counting the number ofpixels in each column or each row of the segmented image that meets thesegmentation criteria.

Step 4: Next, zeroes and maxima are located in the X-axis histogramand/or Y-axis histogram 804. Zeroes and maxima are located by conductinga linear search in the X-axis histogram and/or Y-axis histogram, and thelocations (the numbers of column or row) of the zeroes and maxima arenoted and recorded. Alternatively, other types of searching algorithmsor methods may be employed to locate the zeroes and maxima in the X-axishistogram and/or Y-axis histogram.

Step 5: Use the polynomial regression analyses to determine the shape ofan object according to the zeroes and maxima (805). If the two curveslinking the two zeroes and the maximum in the X-axis histogram and theY-axis histogram are two sloped line, the corresponding object may bedetermined to be a triangle. If each of the four curves linking the twozeroes and the two maxima in the X-axis histogram and the Y-axishistogram is a line, the corresponding object may be determined to be arectangle. If the relation between the two zeroes and the two maxima inthe X-axis histogram and the Y-axis histogram is in the form of aquadratic function, the corresponding object may be determined to be acircle or an ellipse.

Step 6: Determine the numbers of pixels in the X axis and Y axis fromthe X-axis histogram and the Y-axis histogram and the locations of thezeroes and the maxima. Then, object size may be accurately calculated(806).

Step 7: Determine the location of the object according to the X-axishistogram and Y-axis histogram and the locations of zeroes (807).

Step 8: Repeat from Step 2 to step 7 according to a differentcharacteristic. In this manner, an image may be processed several timesaccording to a plurality of characteristics. Therefore, the goal ofaccurate and fast determination of the shape, size and location of anobject may be achieved.

Now, please refer to FIGS. 2 to 5, which illustrate the first embodimentof the method of the present invention. The first embodiment includesthe following 8 steps.

Step 1: As illustrated in FIG. 2, a digital color image which containsfive objects (a red triangle 1, a red circle 2, a red rectangle 3, agreen triangle 4 and a blue rectangle 5) are shown. Such image may begenerated by a computer or a digital or analog image capture device.

Step 2: As illustrated in FIG. 3, only red objects (the red triangle 1,red circle 2 and red rectangle 3) are kept and objects of other colors(the green triangle 4 and blue rectangle 5) are removed during thesegmentation process.

Step 3: Calculate the X-axis histogram and Y-axis histogram. Asillustrated in FIG. 4, the number of red (non-black) pixels in eachcolumn in the segmented image of FIG. 3 is counted to obtain the X-axishistogram. In FIG. 4, the triangular FIG. 101, curved FIG. 201 andrectangular FIG. 301 correspond to the red triangle 1, red circle 2 andred rectangle 3, respectively, in FIG. 3. In FIG. 4, the horizontal axisrepresents column number and the vertical axis represents the number ofpixels. Such X-axis histogram figure contains enough information todetermine the shape, size and location of each object appearing in theoriginal input image in the horizontal direction. As illustrated in FIG.5, the number of red (non-black) pixels in each row in the segmentedimage of FIG. 3 is counted to obtain the Y-axis histogram. In FIG. 5,the triangular FIG. 102, curved FIG. 202 and rectangular FIG. 302correspond to the red triangle 1, red circle 2 and red rectangle 3,respectively, in FIG. 3. In FIG. 5, the horizontal axis represents rownumber and the vertical axis represents the number of pixels. SuchY-axis histogram figure contains enough information to determine theshape, size and location of each object appearing in the original inputimage in the vertical direction.

Step 4: Zeroes and maxima are located within the X-axis and Y-axishistograms. As illustrated in FIG. 4, a linear search is employed tolocate the two zeroes and one maximum and the locations in terms ofcolumn number of these three points are noted and recorded. In FIG. 4,with regard to the triangular FIG. 101, the two zeroes have the heightof zero pixel; the first zero A is located at the 50^(th) column and thesecond zero C is located at the 170^(th) column; also, the maximum B hasthe height of 85 pixels. Then, the four zeroes D, F, G and J and thethree maxima E, H and I are located for the curved FIG. 201 andrectangular FIG. 301. The four zeroes D, F, G and J occur at the273^(rd), 362^(nd), 498^(th) and 563^(rd) columns, respectively. While,the three maxima E, H and I have the same height 86. In the same manner,in FIG. 5, the six zeroes and four maxima are located for the triangularFIG. 102, curved FIG. 202 and rectangular FIG. 302. With regard to thetriangular FIG. 102, the two zeroes K and M occur at the 50^(th) and136^(th) row and the maximum L has the height of 119 pixels. With regardto the curved FIG. 202, the two zeroes N and P occur at the 160^(th) and245^(th) row, respectively, and the maximum 0 has the height of 88pixels. With regard to the rectangular FIG. 302, the two zeroes Q and Toccur at the 280^(th) and 365^(th) row, respectively, and the two maximaR and S have the same height of 65 pixels.

Step 5: Use the polynomial regression analyses to determine the shape ofan object according to the zeroes and maxima (805). As illustrated inFIG. 4, with regard to the triangular FIG. 101, two straight lines 1011and 1012 may be determined according to the two zeroes and one maximum.In the same manner, with regard to the curved FIG. 201, a curve 2011 maybe determined; with regard to the rectangular FIG. 301, three straightlines 3011, 3012 and 3013 may be determined.

After we conduct the polynomial regression analysis for the curve 1011of the object 101 in FIG. 4, we obtain:

Degree 1: −70.28+1.398x, α=0.05, p<0.0001, p<0.0001, R^2=1.00;

Degree 2: −70.71+1.41x−7.0424e−5x^2, α=0.05, p<0.0001, p<0.0001,p=0.6085, R^2=1.00;

Therefore, it is very likely that the curve 1011 is a straight line withthe slope of 1.398 and the result of degree 2 is statisticallyinsignificant at α=0.05.

After we conduct the polynomial regression analysis for the other curve1012 of the triangular FIG. 101 in FIG. 4, we obtain:

Degree 1: 237.2−1.398x, α=0.05, p<0.0001, p<0.0001, R^2=1.00

Degree 2: 238.5−1.418x+7.0424e−5x^2, α=0.05, p<0.0001, p<0.0001,p=0.6085, R^2=1.00

Therefore, it is very likely that the curve 1011 is a straight line withthe slope of 1.398 and the result of degree 2 is statisticallyinsignificant at α=0.05.

Consequently, we can determine that the first object 101 in FIG. 4 is ared triangle.

Then, after we conduct the polynomial regression analysis for the curve2011 of the object 201 in FIG. 4, we obtain:

Degree 1: 48.71+0.05037x, α=0.05, p=0.0784, p=0.5589, R^2=0.00;

Degree 2: −3331+21.48x−0.03375x^2, α=0.05, p<0.0001, p<0.0001, p<0.0001,R^2=0.95;

Therefore, it is very likely that the object 201 is a circle or anellipse and the result of degree 2 is statistically significant atα=0.05.

Consequently, we can determine that the second object 201 in FIG. 4 is ared circle.

Next, with regard to the object 301, because the three curves 3011, 3012and 3013 are vertical or horizontal straight lines, the polynomialregression analysis can not be conducted. Moreover, because the lines3011 and 3013 represents the line linking an zero with a maximum and theline 3012 represents the line linking the two maxima, we conclude thatthe object 301 is a red rectangle 3.

Step 6: After we determine the shapes of these objects, we proceed todetermine the sizes of them. With regard to the triangular FIG. 101 inFIG. 4, we have obtained that the two zeroes A and C occur at the50^(th) and 170^(th) column and the maximum B has the height of 85pixels, and hence the actual size of the red triangle 1 may becalculated from the characteristics of the camera and images and thelocation of the camera.

With regard to the curved FIG. 201 in FIG. 4, we have obtained that thetwo zeroes D and F occur at the 273^(rd) and 362^(nd) column and themaximum E has the height of 86 pixels, and hence the actual size of thered circle 2 in FIG. 2 may be calculated from the characteristics of thecamera and images and the location of the camera.

With regard to the rectangular FIG. 301 in FIG. 4, we have obtained thatthe two zeroes G and J occur at the 498^(th) and 563^(rd) column andboth of the two maxima H and I have the same height of 86 pixels, andhence the actual size of the red triangle 3 in FIG. 2 may be calculatedfrom the characteristics of the camera and images and the location ofthe camera.

Step 7: Now, we calculate the location of each of these objects. Pleaserefer to FIGS. 4 and 5. We have obtained six zeroes (A, C, D, F, G andJ) in FIG. 4 and six zeroes (K, M, N, P, Q and T) in FIG. 5. With regardto the triangular FIG. 101 in FIG. 4, we have obtained that the twozeroes A and C occur at the 50^(th) and 170^(th) column in the X-axis ofFIG. 3 and that the two zeroes K and M occur at the 52^(nd) and 136^(th)row in the Y-axis of FIG. 3. Therefore, the location of the red triangle1 may be obtained.

With regard to the curved FIG. 201 in FIG. 4, we have obtained that thetwo zeroes D and F occur at the 273^(rd) and 362^(nd) column in theX-axis of FIG. 3 and that the two zeroes N and P occur at the 160^(th)and 245^(th) row in the Y-axis of FIG. 3. Therefore, the location of thered circle 2 may be obtained.

With regard to the rectangular FIG. 301 in FIG. 4, we have obtained thatthe two zeroes G and J occur at the 498^(th) and 563^(rd) column in theX-axis of FIG. 3 and that the two zeroes Q and T occur at the 280^(th)and 365^(th) row in the Y-axis of FIG. 3. Therefore, the location of thered rectangle 3 may be obtained.

The actual size of each the objects in FIG. 3 may be calculated from thecharacteristics of the camera and images and the location of the camera.

Step 8: Repeat step 2 to 7 to obtain the shape, size and location of thegreen object or the blue object.

In comparison to the methods of the prior art, the method of the presentinvention may be carried out without any pattern matching database andpattern matching; therefore, the method of the present invention isfaster and more accurate in object recognition. In addition, objectshape information is derived using the polynomial regression analysis,which also provides a measure of shape recognition accuracy, based uponR^2 fit values. Moreover, objects are located without direct input imagesearching, which is inaccurate and time-consuming.

Now, please refer to FIGS. 6 to 9 for the second embodiment of themethod of the present invention. The second embodiment of the method ofthe present invention is mainly used in the recognition of traffic signsand includes the following steps:

Step 1: As illustrated in FIG. 6, an image is acquired by an imagecapture device. The image contains a traffic sign 6. The traffic signincludes a red outer ring 61, white inside 62 and a black number 63. Inaddition, the image has other red objects, such as the red flowers 71 ofthe tree and the red blossoms 71 of the bushes. Some of the red flowers71 are blocked by the green leaves.

Step 2: As illustrated in FIG. 7, only the red objects (the red outerring 61, the red flowers 71 of the tree and the red blossoms 71 of thebushes) are chosen and objects in other colors are removed in thesegmentation process.

Step 3: The number of red pixels in each column and row in the segmentedimage of FIG. 7 is counted to obtain the X-axis histogram (FIG. 8) andthe Y-axis histogram (FIG. 9), respectively. Either FIG. 8 or 9 containsa centrally concave pattern 501 (which corresponds to the outer ring61), a red flower pattern 502 and a red blossom pattern 503, whichcorresponds to the blossoms 71 of the bushes. In FIG. 8, we can see thatthe centrally concave pattern 501 has a distinctive shape and does notoverlap with the other two patterns 502 and 503. While, in FIG. 9, theouter ring 61 and the red flowers 71 overlap in the vertical axis ofFIG. 7 and the latter is considered as noise because the former is whatwe are aiming for. Similarly, in FIG. 9, the centrally concave pattern501 has a distinctive shape.

Step 4: Scanning is carried from the 0^(th) column to the 3264^(th)column and from the 0^(th) row to the 3264^(th) row in FIG. 7 to obtainFIGS. 8 and 9. With regard to the centrally concave pattern 501, twozeroes A′ and D′ and two maxima B′ and C′ are located in FIG. 8 and twozeroes E′ and H′ and two maxima F′ and G′ are located in FIG. 9; also,the four zeroes A′, D′, E′ and H′ occur at the 1112^(th) column,2040^(th) column, 1099^(th) row and 2078^(th) row, respectively;moreover, both of two maxima B′ and C′ have the same height of 710pixels, and the two maxima F′ and G′ have the height of 660 and 650pixels, respectively. In addition, the method used to search for thefour zeroes A′, D′, E′ and H′ and the four maxima B′, C′, F′ and G′ isthe same with the one used for FIGS. 4 and 5.

Step 5: Then, we conduct the polynomial regression analysis for thethree curves 5011, 5012 and 5013 in FIG. 8 to determine the shape of theobject.

The result for the curve 5011 from the polynomial regression analysisis:

Degree 1: −4439+4.145x, α=0.05, p<0.0001, p<0.0001, R^2=0.93

Degree 2: −42765+68.91x−0.02733x^2, α=0.05, p<0.0001, p<0.0001,p=0.0001, R^2=0.99

Therefore, it is very likely that the curve 5011 corresponds to a partof a circle or an ellipse and the result of degree 2 is statisticallysignificant at α=0.05.

The result for the curve 5012 of FIG. 8 from the polynomial regressionanalysis is:

Degree 1: 352.3+0.008745x, α=0.05, p<0.0001, p=0.6111, R^2=0.00

Degree 2: 6456−7.861x+0.002503x^2, α=0.05, p<0.0001, p<0.0001, p<0.0001,R^2=0.89

Therefore, it is very likely that the curve 5012 corresponds to a partof a circle or an ellipse and the result of degree 2 is statisticallysignificant at α=0.05.

The result for the curve 5013 of FIG. 8 from the polynomial regressionanalysis is:

Degree 1: 9155−4.43x, α=0.05, p<0.0001, p<0.0001, R^2=0.92 Degree 2:−102241+109x−0.02887x^2, α=0.05, p<0.0001, p<0.0001, p=0.0001, R^2=0.99

Therefore, it is very likely that the curve 5013 corresponds to a partof a circle or an ellipse and the result of degree 2 is statisticallysignificant at α=0.05.

From the above, we can determine that the centrally concave pattern 501of FIG. 8 is a circle or an ellipse.

Step 6: After we determine the shape of the object, we proceed todetermine its size. As illustrated in FIG. 8, the two zeroes A′ and D′occur at the 1112^(th) and 2040^(th) column and the two maxima B′ and C′has the same height of 710 pixels. While in FIG. 9, the two zeroes E′and H′ occur at the 1099^(th) and 2078^(th) row and the maximum F′ andG′ have the height of 660 pixels and 650 pixels, respectively.Therefore, the actual size of the object may be calculated from thesedata as well as the characteristics of the camera and images and thelocation of the camera.

Step 7: Now, we calculate the location of the object. With the locationsof the four zeroes A′, D′, E′ and H′ (please see FIGS. 8 and 9), we cancalculate the location of the object in the same manner as previouslydescribed.

Step 8: Repeat step 2 to 7 to obtain the shape, size and location ofeach of the objects selected according to other characteristic(s).

The method of the present invention may be used to obtain the color,shape, size and location of an object. In addition, the method may beused for the recognition of traffic signs and in other fields.

With regard to the red flower pattern 502 in FIG. 8, such pattern doesnot have well-defined boundary because the R^2 values of degree 1 anddegree 2 are relatively small in the result of the polynomial regressionanalysis:

Degree 1: −1653+0.5543x, α=0.05, p<0.0001, p<0.0001, R^2=0.48

Degree 2: −155477+102x−0.01672x^2, α=0.05, p<0.0001, p<0.0001, p=0.0001,R^2=0.66

Therefore, the red flower pattern 502 does not correspond to a circularor linear traffic sign.

In addition, noises would not affect the outcome of the polynomialregression analysis. The result for the curve 5014 of FIG. 9 from thepolynomial regression analysis is:

Degree 1: 393−0.02945x, α=0.05, p<0.0001, p=0.0370, R^2=0.01

Degree 2: 5350−6.353x+0.001987x^2, α=0.05, p<0.0001, p<0.0001, p<0.0001,R^2=0.9

Therefore, it is very likely that the curve 5014 corresponds to a partof a circle or an ellipse and the result of degree 2 is statisticallysignificant at α=0.05. This matches what we obtain for the curve 5012 ofFIG. 8. Consequently, the noise (the red flowers 71 of the tree) doesnot affect the outcome in the recognition of the object in FIG. 9.

In comparison to the methods of prior art, the method of the presentinvention has the following two advantages:

1. First, zeroes and maxima are obtained in the x-axis and y-axishistograms. Then, the polynomial regression analysis is conducted.Therefore, the shape, size and location of an object in an image may beswiftly and fast determined.

2. Objects may be selected or segmented according to one of a variety ofcharacteristics, such as a color-specific characteristic (RGB or IHScolor), a characteristic of grayscale (intensity) and a characteristicof visible frequency spectrum. In addition, the shape, size and locationof each of the objects selected may be swiftly and accurately calculatedand determined. In addition, the method may be used in the recognitionof traffic signs and in other fields.

Although a preferred embodiment of the present invention has beendescribed in detail hereinabove, it should be understood that thepreferred embodiment is to be regarded in an illustrative manner ratherthan a restrictive manner, and all variations and modifications of thebasic inventive concepts herein taught still fall within the scope ofthe present invention.

From the above, we can see that the method of the present inventionmeets the relevant patent requirements. It is hoped that the patentapplication will be approved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart including 8 steps for the method of the presentinvention.

FIG. 2 is an image containing five objects in the first embodiment ofthe present invention.

FIG. 3 is a segmented image of FIG. 2 (only the red objects are kept).

FIG. 4 is an x-axis histogram for FIG. 3 by counting the number of redpixels in each column in the x-axis of FIG. 3.

FIG. 5 is a y-axis histogram for FIG. 3 by counting the number of redpixels in each row in the y-axis of FIG. 3.

FIG. 6 is a color image containing several objects for the secondembodiment of the present invention.

FIG. 7 is a segmented image of FIG. 6 (containing only the red objects).

FIG. 8 is an x-axis histogram for FIG. 6 by counting the number of redpixels in each column along the x-axis of FIG. 6.

FIG. 9 is a y-axis histogram for FIG. 6 by counting the number of redpixels in each row along the y-axis of FIG. 6.

List of reference numerals    1 Red triangle    2 Red circle    3 Redrectangle    4 Green triangle    5 Blue rectangle    6 Traffic sign   61Red outer ring   62 White inside   63 Black number   71 Red flowers  101Triangular figure 1011 Curve/straight line 1012 Curve/straight line  102Triangular figure  201 Curved figure 2011 Curve  202 Curved figure  301Rectangular figure 3011 Curve/line 3012 Curve/line 3013 Curve/line  302Rectangular figure  501 Centrally concave pattern 5011 Curve 5012 Curve5013 Curve 5014 Curve  502 Red flower pattern  503 Red blossom patternA, C, D, F, G and J Zeroes B, E, H and I Maxima K, M, N, P, Q and TZeroes L, O, R, S Maxima A′, D′, E′ and H′ Zeroes B′, C′, F′ and G′Maxima

1. A method for recognizing objects in images, comprising steps of: (a)Acquire a digital image; (b) Select one or more objects from the imageaccording to a certain characteristic; (c) Generate an x-axis(horizontal axis) histogram and/or a y-axis (vertical axis) histogramfrom the image of Step (b); (d) Find or locate the zeroes and maxima forthe x-axis histogram and/or the y-axis histogram; (e) Use the polynomialregression analysis to determine the shape of each of the objectsaccording to the zeroes and maxima.
 2. The method as in claim 1, furthercomprises: (a) Select objects from the original image of Step (a)according to another characteristic to generate another image and thenrepeat Step (b) to (e) to process the new image.
 3. The method as inclaim 1, further comprises: (a) Use the coordinates (locations) of thezeroes and maxima to determine the size of the corresponding object. 4.The method as in claim 1, further comprises: (a) Use the coordinates(locations) of the zeroes and maxima to determine the location of thecorresponding object.
 5. The method as in claim 1, wherein the digitalimage of Step (a) is acquired by a digital image capture device.
 6. Themethod as in claim 5, wherein the digital image capture device is astill camera or a video camera.
 7. The method as in claim 1, wherein, inStep (a), an analog image is first acquired by an analog image capturedevice and such analog image is then converted to a digital image. 8.The method as in claim 7, wherein the analog image capture device is astill camera, a video camera or other types of image capture devices. 9.The method as in claim 1, wherein, in Step (b), the characteristic is anRGB or IHS color characteristic.
 10. The method as in claim 1, wherein,in Step (b), the characteristic is a characteristic of grayscale. 11.The method as in claim 1, wherein, in Step (b), the characteristic is acharacteristic of visible frequency spectrum.
 12. The method as in claim1, wherein, in Step (c), the x-axis histogram and the y-axis histogramare generated by counting the number of pixels in each column and eachrow, respectively, of the segmented image of Step (b).
 13. The method asin claim 1, wherein, in Step (d), the zeroes (the points with the heightof zero pixel) are located by conducting a linear search in the x-axishistogram and/or the y-axis histogram.
 14. The method as in claim 1,wherein, in Step (d), the maxima (the points with the maximal height)are located by conducting a linear search in the x-axis histogram and/orthe y-axis histogram.
 15. The method as in claim 1, wherein, in Step(d), the coordinates (locations) of the zeroes and maxima are noted andrecorded.
 16. The method as in claim 1, wherein, in Step (e), in the useof the polynomial regression analysis, if the two curves linking twozeroes and one maximum in the x-axis histogram or the y-axis histogramare two sloped line, the corresponding object may be determined to be atriangle.
 17. The method as in claim 15, wherein, if the four curveslinking two zeroes and two maxima in the x-axis histogram or the y-axishistogram are a straight line, the corresponding object may bedetermined to be a rectangle.
 18. The method as in claim 15, wherein, ifthe relation between two zeroes and two maxima in the x-axis histogramor the y-axis histogram is in the form of a quadratic function, thecorresponding object may be determined to be a circle or an ellipse. 19.A method for recognizing objects in images, comprising the followingsteps: (a) Acquire a digital image; (b) Select one or more objects fromthe image according to a certain characteristic; (c) Generate an x-axishistogram and/or a y-axis histogram from the image of Step (b); (d) Findor locate the zeroes and maxima for the x-axis histogram and/or they-axis histogram; (e) Use the polynomial regression analysis todetermine the shape of each of the objects according to the zeroes andmaxima; (f) Use the coordinates (locations) of the zeroes and maxima todetermine the size of the corresponding object.
 20. A method forrecognizing objects in images, comprising the following steps: (a)Acquire a digital image; (b) Select one or more objects from the imageaccording to a certain characteristic; (c) Generate an x-axis histogramand/or a y-axis histogram from the image of Step (b) (d) Find or locatethe zeroes and maxima for the x-axis histogram and/or the y-axishistogram; (e) Use the polynomial regression analysis to determine theshape of each of the objects according to the zeroes and maxima; (f) Usethe coordinates (locations) of the zeroes and maxima to determine thesize of the corresponding object; (g) Use the coordinates (locations) ofthe zeroes and maxima to determine the location of the correspondingobject.